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0\ ', Abstract 

We review and develop a selection of models of systems with competition and 
cooperation, with origins in economics, where deep insights can be obtained by the 
^-s • mathematical methods of game theory. Some of these models were touched upon 
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the necessary background on games can be found. 
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in : 

f — . We review and develop below some models of competition and cooperation, with 
1 J origins in economics. These models are linked only ideologically and methodologically, 
^ which allows one to read all sections almost independently. 

<n : 

^ ; 1 Territorial price building 

•rH . 

Here we present a variation of the classical Cournot model of price building, where selling 
03 ; and production sites are spatially distributed. 

Suppose there is a market that include selling sites Mi, . . . , M m , products 1, . . . , K 
and production sites 1, L. 

There are N agents that buy the products at the production sites and sell them in 
M l5 ...,M m . 

Assume that at the each production site I there is a large stock of all products available 
at the price pki for the unit of the product k, and that the transportation cost to Mj of a 
unit of the product k is 

Let denote the amount of the product k, which the agent n brought to Mj from 
the site /. Then the total amount of kth product in Mj is 



N 

Yik = Ylki ^ik = Yw- 



n=l 1=1 
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The selling price of the product k in Mi is the function of the total supply of the product. 
Following the standard Cournot model assume that this price is given by the formula 

Rik(Xik) = (1 — Y ik /aik)flik, Y ik G [0, oijfc], 
with some positive constants aci k , 0i k . 

Then the total income of the player n can be calculated by the formula 

m K 
i=l k=l 

with 



H n (ik) = Yf k (l - ^) ik - T Y^Mm + Ph), Y lk e [0, a ik 



(2) 



Thus we have defined a symmetric A^-person game T N with the payoffs given by (CEJ) and 
with the strategies of each player n being the arrays (Y ikl ) with positive entries bounded 
by a ik . 

For simplicity we reduce our attention to the case of 2 players. 

Proposition 1.1. The game V 2 has a symmetric Nash equilibrium. If for each pair (i, k) 
there exists a unique I = q(i, k) where the minimum of + p k i is attained, then this 
equilibrium is unique and the corresponding strategies of each player are given by 

Y ik = 2°q(i,k) a ik I 1 7^ I , (3) 

where 5 l m denotes the usual Dirac symbol that equals 1 for I = m and zero otherwise. 

Proof. Clearly looking for a best reply one has to maximize each H n (ik) separately for 
each pair (ik). It is also clear from fl2]) that to have a best reply one has to bring the 
product only from those sites that minimize ^ ik i + p k \. Hence given {Ylk) ^° ^ n( ^ ^ ne ^ es ^ 
reply of the second player one has to find the maximum of the functions 

/ y.i _|_ y? h . k \ 

^2 \Yikq{i,k)) = *ifcg(i,fc)) 1 ~ " I Pik ~ (£ikq{i,k) + Pkq(i,k)) 

\ Ctik J 

for each pair (ik). Differentiating one gets 

dH?{z) / Yl + z\ Q ,, _ . 8^ 



dz 

— T. 

ikq(i,k) 



( 1 — J Pik — (£ikq(i,k) +Pkq(i,k)) — Z— = 

V a ik J OL ik 



for the best reply z = Y? ha(i k) , yielding 



V 2 1 , ( -i Y^ k £ikq(i,k) + Pkq(i,k)\ /.n 

Y ikqm - - 2 a lk j . (4) 

As the second derivatives of H!jr is negative this point does really define a maximum (and 

not a minimum). By the symmetry one concludes that symmetric equilibrium strategy 
solves the equation 

x> 1 /-, %k £ikq{i,k) + Pkq(i,k)\ / e x 

Y ik = -a lk 1 - (5) 

2 V a ik p ik I 

and thus equals d3J) as required. □ 
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2 Bi-matrix inspection games 



In this and the next sections we develop some extensions of the classical inspection game. 
For other similar models see pQ, [2], [5], [10] and references therein. 

The game is carried out between a trespasser (player I) and an inspector (player II) 
in n-periods (or steps). We shall analyze a 'stop to abuses' version of this game when it 
lasts only till the first time trespassing is discovered by the inspector. 

Player I has 2 pure strategies: to break (violate) the law (B) or to refrain from it (R). 
Player II has also 2 pure strategies: to check the actions of player I (C) or to have a rest 
(R). If player I chooses (R) he gets the legal income r > 0. If he chooses (B), he obtains 
additionally the illegal surplus s > 0. However, if his illegal action is discovered by player 
II, the player I pays the fine / > and the game is over. 

If player II chooses (C) he spends the amount c > on this procedure and can discover 
the trespassing of player I with the probability p(p = 1 — p). If player I breaks the law 
and this action is not discovered, the inspector loses the amount I > 0. 

Consequently one step of this game can be described by the table 



Inspector 



Trespasser 





Check (C) 


Rest (R) 


Break (B) 


—pf + p( r + s )> ~( c + pl) 


r + s, — I 


Refrain (R) 


r, —c 


r,0 



(6) 



or shortly by the matrix 

-pf + p(r + s), — (c + pl) r + s,—l 

r, — c r, 

It is natural to assume, that c < pi, so that the pair (B,R) is not a Nash equilibrium 
(otherwise the inspector has no reasons at all to conduct checks). 

Let us say that a bi-matrix game has a value, if payoffs in any of the existing Nash 
equilibrium are the same, the corresponding pair of payoffs being called the value of the 
game. 

Proposition 2.1. Assume f, r, s,c,l > 0, c < pi. Then the game with the matrix ([6]) has 
a value V = (u,v). Moreover, 1) if s < s\ — |(/ + r), the unique Nash equilibrium is 
given by the pair of mixed strategies (x,x), (y,y), where 



x 



u 



r, v = — - 



pi p(/ + r + s)' p 
2) if s > Si, the unique Nash equilibrium is the profile (B,C) and 

-pf + p(r + s), v — — (c + pi); 



u 



3) if s = s\, then u = r,v = — (c + pi) and the Nash equilibria are given by all pairs 
(X,C), where X is any (pure or mixed) strategy of the trespasser. 

Proof. Clearly the only candidate for Nash equilibrium in pure strategies is the pair 
(B, C). This profile is an equilibrium if and only if s > s\. Formulae for x, y follow from 
the standard expression for mixed strategy profiles in two-action two-player games. Other 
statements are checked by a straightforward inspection. □ 
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Of course of greater interest is the analysis of a multi-step version of this game. Let us 
consider the n-step game, where during this time the player I can break the law at most 
k times and the player II can organize the check at most m times. Assume that after the 
end of each period (step), the result becomes known to both players. Total payoff in n 
steps equals the sum of payoffs in each step. It is also assumed that all this information 
(rules of the game) is available to both players. 

Let us denote the game described above by Y kj7n (n). Let (uk tm (n),Vf. im (n)) be the 
value of this game. We then get the following system of recurrent equations: 

(n),v k (n)) 

/ -p/+p(r + s + tt fc _ lim _i(n-l)), r + s + u k „i >m (n - 1), \ 
= VaH -(c + pl) +pv k -i im -i(n- 1) -I + v k -i, m (n - 1) (7) 

V r + u k , m -i(n - 1), -c + v ktJn -i{n - 1) r + u ktm (n - 1), v ktm {n - 1) / 

(if all T ktTn (n) have values, i.e. their equilibrium payoffs are uniquely defined), with the 
boundary conditions (m,n, k > 0): 

(tto, m (n),«o,m(n)) = (nr,0); (8) 

(u kfi (n),v k>0 (n)) = (nr + ks, -kl); k<n, (9) 

reflecting the following considerations: if the trespasser is unable to break the law, the 
pair of solutions (R,R) will be repeated over all periods; and if the inspector is unable to 
check, the trespasser will commit the maximum number of violations available. 

Though k < n,m > n, the form of the recurrent equations below is slightly simplified 
if one allows all non-negative k, m, n together with the agreement 

(uk,m(n),v k , m (n)) = (ufc', m '(n),u fc ' im '(n)); k' = mm(k,n),m' = min(m,n) (10) 

Let us reduce our attention further to the game r nn (n). Let us write U n = u ntn (n) 
and V n = v ntn (n). Then ([7]) takes the form: 

(U n , V n ) = ([/„_!, K-i) + ValM n , (11) 

where 

M = f p(r + s) -p(f + U n ^ 1 ),-(c + pl+pV n _ 1 ) r + s,-l\ ^ 

{n > 0; U = V = 0) 

For n = 1 the game becomes the same as the game ([6]), and its solution is given by 
Proposition 12.11 

Let us find the solution to the game r 2> 2(2). Plugging the values U\ and V\ into M 2 
yields 

f p(r + s) -p(f + r),-pl r + s,-l\ n 
M2 "^ r,-c r,0 J' 0<s<s ^ 

m - ( P(~Pf +P( r + s)),-p(c + pl) r + s,-l\ 

M2 -{ r-c r,0 J' S>Sl ' 
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S2 = =(/ + r) + -jr. 
p p 

Direct calculations show that under the assumptions of Proposition 12.11 the M 2 also 
has a value, and one can distinguish three basic cases (equilibrium strategies are again 
denoted by (x,x), (y,y)): 

1) if < s < si, then 

c s c{2pl + c) 

x = 7 , :V= tt , n — ; — \\ U 2 = 2r,V 2 - 



pl + c p(f + 2r + s) ' p(pl + c 

2) if si < s < s 2 , then 



c 



p(c+(l+J>)Z) y p(p/ + (l+p)(r + s))' 

c/ 

U 2 = -pf+p(r + s), V 2 = -(———— — + c + pi) 

p(c+ (1 +p)t) 

3) if s > s 2 , then 

U 2 = (l+p)(-p/ + p( r + s)), V 2 = -(1 +p)(c + pO, 

and the profile (B,C) is an equilibrium. 

Analogously one can calculate the solutions for other n > 2. 



3 Tax payer against a tax man 

This is a game between a tax payer (player I) and the tax police (player II). Player I has 
2 pure strategies: to hide part of the taxes (H) or to pay them in full (P). Player II has 
also 2 strategies: to check player I (C) and to rest (R). Player I gets the income r if he 
pays the tax in full. If he chooses the action (H), he gets the additional surplus I. But if 
he is caught by player II, he has to pay the fine /. 

In the profile (C,H) player II can discover the unlawful action of player I with the 
probability p(p = 1 — p), so that p can be called the efficiency of the police. Choosing 
(C), player II spends c on the checking procedure. Of course /, r, f,c> 0. 

Hence we defined a bi-matrix game given by the table 



Player II (Police) 





Check (C) 


Rest (R) 


Hide (H) 


r + pi — pf, —c + pf — pi 


r + l,-l 


Pay (P) 


r, —c 


r,0 



or shortly by the payoff matrix 

/ r +pl — pf, —c + pf — pi r + l,—l 
I r, — c r, 

The candidates to the mixed equilibrium are the strategies ((3,(3), (a, a), where 

a 22 ~ 0-12 I n 
Ct = = —r, -pr > 

an ~ a 12 - a 21 + a 22 p{l + J) 
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In order to have these strategies well defined, it is necessary to have a < 1 and (3 < 1 
respectively. By a direct inspection one gets the following result. 



Proposition 3.1. 1) If c> p(f + I), the pair (H,R) is an equilibrium, and moreover the 
strategy (R) is dominant for the police (even strictly, if the previous inequality is strict). 
2) If c < p(f + I) and fp < pi, the pair (H,C) is an equilibrium and the strategy (H) is 
dominant (strictly if the previous inequality is strict). 3) If c < p(f + I), fp > pi, then 
the unique Nash equilibrium is the profile of mixed strategies ((3,(3), (a, a). 

Remarks. 1. Consequently, in cases 1) and 2) the actions of the police are not 
effective. 2. It is not difficult to show that the equilibrium in case 3) is stable. 

It is more interesting to analyze the game obtained by extending the strategy space 
of player I by allowing him to choose the amount I of tax evasion: I G [0, Im], where Im 
is the full tax due to player I. For example, we shall assume that the fine is proportional 
to /, i.e. f{l) = nl. Say, in the Russian tax legislation n = 0.4. Under these assumptions 
the key coefficients a, (3 take the form 

1 a C 1 

a = ~ 1 T\i P = 7" 



p(n + 1) ' I p(n + 1) 

Let Hi (I) denote the payoff to player I in the equilibrium when I is chosen. One can 
distinguish two cases: 

l>l i = , °, rr ^ (3<l, 
p{n + 1) 

then ((3,(3), (a, a) is a stable equilibrium. If I < li <^=^ (3 > 1, then (H,R) is a stable 
equilibrium. Since 

Hi(l < h) = r + I < r + h, 

Hi(l > h) = (3a(r + pl — pf) + (3a(r + I) + (3ar + (3ar = r + (3l(ap + a — apn), 

it follows that Hi(l > l\) > Hi(l < li) would be possible whenever (3l(ap + a — apn) > l\. 
However ap+a—apn = 0, hence this is not the case. Consequently H T (l > li) < Hf(l < Zx) 
and therefore player I will avoid tax on the amount I — l±. 
2) p < 4? a>l. 

If / > li <^=^ (3 < 1, then (H, C) is an equilibrium, and if I < h /3 > 1, then (H, R) 
is an equilibrium. Since 

H^l < k) = r + I < r + l±, H^l > h) = r + (1 - p)l - pin, 

the choice / > l\ is reasonable for player I as 

1(1 — p — pn) > l\. 

Hence Hj(l > l x ) > Hj(l < h) whenever / > 1 _ p l ( i n+1) ■ 
Consequently, if 

U 



the equilibrium strategy for player I is I = Im and otherwise I = l\. 

One can conclude that in both cases it is profitable to avoid tax on the amount l±, but 
as the efficiency of tax man increases, it becomes unreasonable to avoid tax on a higher 
amount. 

Let us see which condition in the second case would ensure the inequality f)14p when 
the amount of tax avoidance is Im in the equilibrium. Plugging 1% in ( 114)) yields 



p(n+l)(l-p(n+l)) 
Denoting x = p(n + 1) < 1 one can rewrite it as 



< / 



M- 



X 



x + — < 0. 

Im 



(15) 



The roots of the corresponding equation are 

1 ± 

Xl,2 = 



1_ 4c 



Hence for c > /m/4 inequality (fl4"]l does not hold for any p, and for c < /m/4 the solution 
to ([15D is 



Thus for 



x G 



, hi 



l 



l 



1 + 



p G 



4c 



1 + 



4c 



2(n + 1) 



2n 



(16) 



it is profitable to avoid tax payment on the amount Im- 

Let us consider a numeric example with n = 0.4 so that l/(n + 1) = 0.714. If, say, 
c = 1000, then l M = 100000. 

1) If p < 0.714, the condition c < / M /4 holds true, as 1000 < 2500. Hence, for 
p G [0.007; 0.707] it is profitable to avoid tax on the whole amount, i.e. 100000. 

2) If p > 0.714 it is profitable to avoid tax on the amount l\ = 714.29. 

Hence if the efficiency of tax payment checks is p < 0.707, it is profitable to avoid tax 
on the whole amount of 100000, and if p > 0.707, then not more than on 1010. 



4 Cooperative games versus zero-sum games 

The aim of this Section is to present a curious connection between competition and 
cooperation by linking the solutions to cooperative games and lower values of auxiliary 
zero-sum games. The following definitions are standard: 

Cooperative game with non-transferable utility is a triple G = (I,v,H), where I = 
{1,2, . . . ,n} is the set of players, if is a non-empty compact set from R n , and v is a 
mapping from the set of all coalitions (non-empty subset S C I) to the set of non-empty 
closed subsets v (S) C H. 
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For x,y G H one says that x dominates y (x y y), if there exists a coalition S C. I 
such that 

1) x,y G u(S'); 

2) Xj > yj for any i E S. 

A (von Neumann - Morgenstern) solution to G = (I,v,H) is called a subset V C if 
such that 

1) (internal stability) there are no pairs of vectors from V such that one of them 
dominates the other one; 

2) (external stability) for any y G if\V there exists x & V such that x y y. 

For A C i? n let us denote by A £ the e-neighborhood of A, i.e. A £ = A + B £ with 



B £ = { x (z R r ' 



i=i 



A closed subset V is called a e-solution whenever is internally stable and for any 
y G H\V e there exists x (z V such that x y y. 

It is clear that if a closed A is a e-solution in the game G = (I, v, H) for any e > 0, 
the A is a solution. 

Consider the function 

L(x, y) = max min (xj — 

S:x,yev(S) ieS 

It is clear that L(x, y) > <^> x >- y. Let ^4(G) = {B e 2 H : B , max L(x, j/) = 0}. 

x,y£B 

Proposition 4.1. Lei e > 0. The game G = (I, v, H) has a e-solution if and only if 

sup { min maxL(x, y)} > 0. (17) 

AtA(G) y&H\Ae xeA 

Proof. Let (ITT)) holds. Then there exist A C if such that 

min maxL(x,y) > (18) 



and 



max L(x, y) = 0. (19) 

x,ydA 



It follows from (ITS]) that maxL(x, y) > for each y G H\A e . Consequently for arbitrary 

x£A 

y G H\A £ one can find a x G A such that L(x, y) > 0, i.e. x y y. It now follows from ( TT9|) 
that L(x,y) < for all x,y e A, i.e. neither x dominates y, nor vice versa. Consequently 
the set A is a e-solution. 

Now let ^4 is a e-solution. Then L(x,y) < for all x,y £ A, but L(x, x) = implying 

(USD- 

With A being a e-solution, for any y G H\A £ there exists a x G A such that x y y, 
i.e. L(x,y) > 0. Consequently maxL(x, y) > for any y G H\A £ and therefore 

min maxL(x, y) > 0. 

y£H\A £ xeA 

□ 
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Let us choose now a e > and introduce a two-person zero-sum game F £ (I,v, H), 
in which the first player makes a first move by choosing a A G *4 and then the second 
player replies by choosing y G H\A £ . Finally the first player makes the third (and the 
last) move by choosing x G A. The income of the first player in this game equals L(x,y) 
(the second one gets —L(x,y)). 

The following statement is a direct corollary of Proposition 14. 11 

Proposition 4.2. The game G = (I,v,H) has a e-solution if and only if the maximal 
guaranteed gain of the first plater in T £ (I, v, H) is positive. Moreover any A G A yielding 
such gain is a e-solution to G. 

One can now define a two-player zero-sum game in the normal form N e (I, v, H), where 
the strategies of the first players are the sets A £ A and the strategies of the second player 
are the mappings / : A G A H\A e . Let F denote the set of the strategies of the second 
player. Let the payoff to the first player in this game N £ (I, v, H) be 

h(AJ) = maxL(xJ(A)). 

Proposition 4.3. The game N £ (I,v,H) has a value, i.e. 

sup inf h(A, f) = inf sup h(A, f). 
AeAf^ F f^ F AeA 

Proof. Consider the strategy /* such that 

min m&xL(x, y) = maxL(x, f*(A)) 

y&H\A £ xEA xeA 



for any A £ A. 
Then 



sup inf h(A, f) < inf sup h(A, f) < sup h(A, f* 

AeAf eF f£ p AeA AaA 



sup maxL(x, f*(A)) = sup min maxL(x, y) = sup inf h(A, f). 

AeA x(iA AeAV^ H \ A e zeA AeA f£F 



□ 



Theorem 4.1. (fU^/) The game G = (I,v,H) has a e-solution if and only if 

sup inf h(A, f) > 
AeAf^ F 

holds for the game N e (I, v, H). 

Proof. Follows from Proposition I4.1[ 14.31 □ 



5 Stability for two-action multi-agent games 

One of the most popular game in recent game-theoretic literature is the so called minority 
game, see [3]. It represents a particular game of many agents with each agent having two 
strategies. With this motivation, we are going to analyze here the stability property of 
games with two actions of each player. 
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To prepare the stage for our analysis, let us consider a general non-symmetric game 
r of m agents, where each agent j has rij strategies {sj}, k = 1, • • • ,rij, and receives 

payoffs Uj(s J i, s J 2 2 , ■ ■ ■ ,s^) at a profile (s-f , S2 2 , ■ • • ,s^). Then, for a mixed strategy 
profile a — (a%, cr 2 , a m ) with 

ci = •••) x n -i)i a 2 = (xi, x n2 ), a m = (x 1 , x nm ) 

the payoff of agent j becomes 

ni ri2 n m 

ii^i-^ o = EE - E •'•!:4-< II '(-r-^ c)- (20) 

31=1.72=1 jm=l 

The Replicator Dynamics (RD) equations have the form 

x l = ( n i( s }' a -i) ~ II ,(>'.;• n , ))./•;. i = 1, ...,nj, j = 1, ...,m, (21) 

where er_ 3 - denotes the collection of the strategies of all players in a others than j. This 
system describes an evolution of the behavior of the players applying a try-and-error 
method of shifting the strategies in the direction of a better payoff. As is well known, a 
Nash equilibrium is a fixed point of system f[2~TT) . The r.h.s of ( 121 j) is sometimes called the 
Nash vector field of the game. 

Recall that a Nash equilibrium for a game T is called asymptotically stable, neutrally 
stable or unstable in the Lyapunov sense (or dynamically) if it so for the corresponding 
dynamics ( f2T|) . (This means roughly speaking that if starting with strategies near the 
equilibrium, players would adjust their strategies in the direction of better payoffs, their 
strategies would converge to this equilibrium.) On the other hand, a Nash equilibrium 
(o"i, ...,cr m ) in such game is called structurally stable if for arbitrary e > there exists a 
5 > such that for all games T with the same number of players and pure strategies and 
with payoffs II that differ from II no more than by 5, i.e. such that 

| Il^af , 4 2 , . . . , st ) - IT (sf , 4 2 , . . . , st )\<8 (22) 

for all i, s?i, s J 2 2 , s{^, there exists a Nash equilibrium (<7i, a m ) for the game T such 
that —o~j | < e for all j = 1, m. It makes sense also to speak about structural stability 
of dynamically stable or unstable equilibria, i.e. a dynamically stable (or unstable) Nash 
equilibrium (01, a m ) in a game T is called structurally stable if for arbitrary e > 
there exists a 5 > such that for all games T with the same number of players and pure 
strategies and with payoffs II that differ from IT no more than by 5 (i.e. ( F22|) holds), there 
exists a dynamically stable (respectively unstable) Nash equilibrium (ax, a m ) for the 
game T such that |<7j — <Jj\ < e for all j = 1, m. 

The notion of stability is closely related to another important notion of a generic 
property: a property (object or characteristics) in a class of structures parametrized by a 
collection of real numbers s from a given subset S of a Euclidean space is called generic 
if it holds for s from a subset S G S that is both open (which means that if So € S, then 
all s G S that are closed enough to s belong to S as well, i.e. the property of being in S 
is structurally stable in any point s E S) and dense (which means that for any so there 
exists an s 6 S that is arbitrary close to so, i.e. the negation of being in S is nowhere 
structurally stable). 
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As mentioned above, we are going to concentrate on the class T\ of mixed strategy 
extensions of games of n players each having only two strategies. 

Let A l j j n denote the payoff to i under pure profile {ji, ...,j n }, jk = 1,2. A mixed 
strategy profile can be described by families 

oi = (xi, 1 - xt), 02 = (x 2 , 1 - x 2 ), a n = (x n , 1 - x n ). 

Equations (12"T|) can be written in terms of xi, ...,x n yielding (check it!) 

Xi = Xi(l-Xi) 2j ^n^II^ - ^' i = l,...,n, (23) 

/6{l,...,n}\i fee/ fc£f 

where 

A i = A i — A i 

I jl-ji-llji+l-jn jl-ji-l2ji+l—jn 

with jk = 1 whenever k G / and = 2 otherwise. Hence pure mixed (i.e. with all 
probabilities being positive) Nash equilibria for a game in are given by vectors x* = 
(x*, ...,x*) with coordinates from (0, 1) solving the following system of n equations 

Yl A i I ]Jx k l[(l-x k ) = 0, z = l,...,n. (24) 

/G{l,...,n}V kel k$I 

In particular for n = 3, denoting xi,x 2 , £3 by x, y, z and arrows of payoffs A 1 , A 2 , A 3 by 
A,B,C yields for system ( 1231) the following explicit form 

i; = x(l - x)(a + A 2 y + A 3 z + Ayz), < x < 1 
y = y(l-y)(b + B lX + B 3 z + Bxz), 0<y<l (25) 
i = z(l - z)(c + Cix + C2y + Cxy), < z < 1 

as well as the form 

a + A 2 y + A 3 z + Ayz = 

b + B x x + B 3 z + Biz = (26) 
c + Cix + C 2 y + Cxy = 

for system (1241) . where 

a = A\ 22 — A 222 , A 2 = A U2 — A 2 i 2 — a, A 3 = A 121 — A 22 i — a, 

A = A U1 - A211 -a-A 2 -A 3 , 

and the coefficients in other two lines are defined analogously. 

Assuming ) solves ([21]), it is convenient to rewrite system (1231) in terms 

of the deviations from the equilibrium £j = X{ — x*. One then sees by inspection that the 
matrix of linear approximation (the Jacobian matrix) J* has the entries 

4=<a-4) Yi {A*-%)n*k n (!-^) ( 2? ) 

/e{l,...,n}\{*.J'} fee/ fc£(/U{i,j}) 

for i j and Jj* = for all i. 

Theorem 5.1. Let J* be the Jacobian matrix (described in Exercise 9.12) of a pure mixed 
equilibrium x* = (x*, •■■,£*) solving (1241) . If at least one of the eigenvalues of J* has a 
non-vanishing real part, then x* is unstable in Lyapunov sense (i.e. dynamically). In 
particular, if n is an odd number, then a necessary condition for the Liapunov stability of 
x* is the degeneracy of J* , that is det J* = 0. 
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Proof. From (1271) one deduces that J* has zeros on the main diagonal. Hence the sum of 
its eigenvalues vanishes. Consequently if there exists an eigenvalue with a non-vanishing 
real part there should necessarily exist also an eigenvalue with a positive real part, which 
implies instability. As eigenvalues with vanishing real parts appear as pairs of conjugate 
imaginary numbers, it follows that in case of odd n the fact that all real parts vanish 
implies that zero should be an eigenvalue, i.e. the degeneracy of J*. □ 

Let us write down the condition det J* = explicitly (i.e. in terms of the payoff 
coefficients) for the case n = 3. 

(i) From (I27p the condition det J* = writes down as 

(A 2 + Az*) (B 3 + Bx*) (d + Cy ) + (B 1 + Bz*) (C 2 + Cx*) (A 3 + Ay") = 0. (28) 

(ii) Solving (|26l) by expressing y, z, in terms of x and putting this in the first equation 
leads to the quadratic equation: 

v{x*) 2 + ux* + w = 0, (29) 

where 

w = aC 2 B 3 + cbA - bA 3 C 2 - cA 2 B 3 , v = aBC + AB X C X - BA 2 C X - CA 3 B U 

u = a{BC 2 + CB 3 ) + b(ACi - CA 3 ) + c{AB x - BA 2 ) - A 2 B 3 C X - A 3 B X C 2 . 

(iii) Using the system that is solved by x*,y*,z* to express y*z* as a linear function 
of y*,z* allows to rewrite ( l28|) as a quadratic equation (in x*,y*,z*). Expressing the 
quadratic terms of this equation again via linear terms leads to the equation on x* only: 

u + 2x*v = 0. 

Comparing this with ( |29i) leads to the conclusion that ( 1281) is equivalent to the equation 

u 2 - Avw = 0, (30) 

which is a polynomial homogeneous equation of the sixth order in coefficients a, A, A 2 , A 3 , 
b, B, £?i, B 3 , c, C, Ci, C 2 . 

Corollary. The property to have a dynamically unstable pure mixed equilibrium is 
generic among the games of type T 2 that have pure mixed equilibria. More precisely, apart 
from the games from the (algebraic of the sixth order) manifold M described by equation 
(1301) pure mixed equilibria are always dynamically unstable and structurally stable (as 
dynamically unstable equilibria). 

Proposition 5.1. Under the assumption ( !30l (or equivalently ( 1281) ) there exist a,/3,7 
such that a function V of the (relative entropy) form 

a[x*lnx+(l-x*)ln(l-a;)]+/3[y*lny+(l-y*)ln(l-y)]+7[z*lnz+(l-z*)ln(l-z)] (31) 

is a first integral for ( |25l) . i.e. it does not change along the trajectories of (125]) (dV/dt = 0) 
if and only if the condition 

AB x Ci + aBC = BA 2 d + CA 3 B X (32) 
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holds. In the latter case 



V = (Bt + Bz*) (d + Cy*) [x* In x + (1 - x*) ln(l - x)] 

-(A 2 + AO(Ci + C7y*)[y*lny + (1 - y*) ln(l - y)] 
-(A 3 + Ay*)(B 1 + Bz*)[z* \nz + (1 - z*) ln(l - z)\ 

is an integral. 

Proof. Substituting (T5TT) in the equation dV/dt = that (13"TI) is an integral of motion 
if and only if 

" a(A 2 + Az) + /3(B 1 + Bz) = 
a{A 3 + Ay)+ 1 {C 1 + Cy) = 
(J(B 3 + Bx)+ 1 (C 2 + Cy)=0 
aA + 0B + 7C = 0. 

Expressing /3,7 in terms of a from the first two equations one observes that the third 
equation is then automatically satisfied due to (125]) . Solving the last equation leads to 

□ 



Having a first integral V as above, allows one to conclude that the equilibrium (x*, y*, z* 
is neutrally stable in the Lyapunov sense. 



6 Nonlinear Markov games 

Nonlinear Markov games arise as a (competitive) controlled version of nonlinear Markov 
processes (an emerging field of intensive research, see e.g. [8], [6] and references therein). 
This class of games can model a variety of situation for economics and epidemics, statis- 
tical physics, and pursuit - evasion processes. 

A discrete-time, discrete-space nonlinear Markov semigroup $ , k G N, is specified by 
an arbitrary continuous mapping $ : S n — > S n , where the simplex 

n 

S n = {fl= (//!, ...,/JLn) E R+ : J^/Xi = 1} 

8=1 

represents the set of probability laws on the finite state space {1, ...,n}. For a measure 
(i G S n the family = can be considered an evolution of measures on {l,...,n}. 
But it does not yet define a random process, because finite-dimensional distributions are 
not specified. In order to obtain a process we have to choose a stochastic representation 
for $, i.e. to write it down in the form 

n 

*(Ai) = {%(/i)}? =1 = {1>«(aOM"=i, (33) 

i=i 

where Pij(fi) is a family of stochastic matrices depending on fi (nonlinearity!), whose 
elements specify the nonlinear transition probabilities. For any given $ : E n 1— > S n a 
representation fl33l) exists but is not unique. There exists a unique representation fl33|) 
with the additional condition that all matrices Pij(fi) are one dimensional: 

PM = ®M, i,j = l,-,n. (34) 
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Once a stochastic representation (1331) for a mapping $ is chosen we can naturally define, 
for any initial probability law \i — //°, a stochastic process ii, I G Z+, called a nonlinear 
Markov chain, on {1, in the following way. Starting with an initial position iq dis- 

tributed according to \i we then choose the next point i\ according to the law {Pj j(/i)}™ =1 , 
the distribution of i\ becoming /i 1 = 

n 

H) = P(ii = j) = J2 p MfM = 

1=1 

Then we choose z 2 according to the law {■Pi 1J -(/jt 1 )}" = i, and so on. The law of this process 
at any given time k is fi k = that is, it is given by the semigroup. However, now 

the finite-dimensional distributions are defined as well. Namely, say for a function / of 
two discrete variables, we have 

n 

Ef(i kl i k+1 ) = J2 f(hj)Vi P ij(v k )- 
i,i=i 

In other words, this process can be defined as a time nonhomogeneous Markov chain with 
transition probabilities Pij([i k ) at time t = k. 

We turn now to nonlinear chains in continuous time. A nonlinear Markov semigroup in 
continuous time and with finite state space {1, n} is defined as a semigroup t > 0, 
of continuous transformations of S n . As in the case of discrete time the semigroup itself 
does not specify a process. To get a process, assume the semigroup $' is differentiable in 
t, so that we can define the (nonlinear) infinitesimal generator of the semigroup as the 
nonlinear operator on measures given by 

The semigroup identity for $* implies that solves the Cauchy problem 

±&(ja) = A(&M), $°(/i) = /i. (35) 

As follows from the invariance of £„ under these dynamics, the mapping A is con- 
ditionally positive in the sense that m = for a fi G S n implies Ai(p) > and is also 
conservative in the sense that A maps the measures from E n to the space of signed mea- 
sures 

n 

£° = {z, G R" : ;>> = 0}. 

8=1 

We shall say that such a generator A has a stochastic representation if it can be written 
in the form 

n 

A j{y) = ^2viQij(v) = (pQ(p))j, ( 36 ) 
»=i 

where Q(fi) = {Qijif-)} is a family of infinitesimally stochastic matrices (or Q=matrices) 
depending on fi G E n . Thus in its stochastic representation the generator has the form of a 
usual Markov chain generator, though depending additionally on the present distribution. 
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The existence of a stochastic representation for the generator is not difficult to obtain, 
see [8]. 

The examples of nonlinear Markov chains are numerous including Lotka-Volterra sys- 
tems, general replicator dynamics of the evolutionary game theory, models of epidemics, 
coagulation processes, see more in [8]. 

Now we discuss the corresponding nonlinear extension of controlled processes. 

Nonlinear Markov games can be considered as a systematic tool for modeling decep- 
tion. In particular, in a game of pursuit - evasion, an evading object can create false 
objectives or hide in order to deceive the pursuit. Thus, observing this object leads not 
to its precise location, but to its distribution only, implying that it is necessary to build 
competitive control on the basis of the distribution of the present state. Moreover, by 
observing the action of the evading objects, one can make conclusions about its certain 
dynamic characteristics making the (predicted) transition probabilities depending on the 
observed distribution via these characteristics. This is precisely the type of situations 
modeled by nonlinear Markov games. 

The starting point for the analysis is the observation that a nonlinear Markov semi- 
group is after all just a deterministic dynamic system (though on a weird state space 
of measures and with a specifically structured payoff function). Thus, as the stochastic 
control theory is a natural extension of the deterministic control, we are going to further 
extend it by turning back to deterministic control, but of measures, thus exemplifying the 
usual spiral development of science. The next 'turn of the screw' would lead to stochastic 
measure-valued games forming a stochastic control counterpart for the class of processes 
discussed in the previous section. 

We shall work directly in the competitive control setting (game theory), which of 
course includes the usual optimization as a particular case, but for simplicity only in 
discrete time and finite original state space {1, ...,n}. The full state space is then chosen 
as a set of probability measures S n on {1, ...,n}. 

Suppose we are given two metric spaces U, V of the control parameters of two players, 
a continuous transition cost function g(u,v,fi), u G U, v G V, fi G S n and a transition 
law z/(u,t>,/i) prescribing the new state v G E n obtained from \i once the players had 
chosen their strategies u G U, v G V. The problem of the corresponding one-step game 
(with sequential moves) consists in calculating the Bellman operator 

(BS)(fJ,) = minmax[g('u, v, /i) + Siy^u, v, /i))] (37) 

U V 

for a given final cost function 5 on E n . According to the dynamic programming principle 
(see e.g. [10J ) , the dynamic multi-step game solution is given by the iterations B k S. Often 
of interest is the behavior of this optimal cost B h S(n) as the number of steps k go to 
infinity. 

The function i>(u,v,fi) can be interpreted as the controlled version of the mapping $ 
specifying a nonlinear discrete time Markov semigroup. Assume a stochastic representa- 
tion for this mapping is chosen, i.e. 

■n 

Uj(u, V,/J,) = ^ HiPij(u, V, fj) 

1=1 

with a given family of (controlled) stochastic matrices P^. Then it is natural to assume 
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g to describe the average over the random transitions, i.e. be given by 



g(u,v,/i) = ^2 ^Pij(u,v,/j,)g lj 

with certain real coefficients gij. Under this assumption the Bellman operator ( 157)) takes 
the form 

n / n \ 

(BS)(ji) = mmmax[^2 fj li Pij(u,v,fi)g i j + S I ^/^.(m, ?;,//) J]. (38) 

V i,j=l \i=l J 

We can now identify the (not so obvious) place of the usual stochastic control theory 
in this nonlinear setting. Namely, assume Py above do not depend on \x. But even then 
the set of the linear functions S{p) = Y17=i Si f jl on measures (identified with the set of 
vectors S = (si, s n )) is not invariant under B. Hence we are not automatically reduced 
to the usual stochastic control setting, but to a game with incomplete information, where 
the states are probability laws on {l,...,n}, i.e. when choosing a move the players do 
not know the position precisely, but only its distribution. Only if we allow only Dirac 
measures /i as a state space (i.e. no uncertainty on the state), the Bellman operator would 
be reduced to the usual one of the stochastic game theory: 



n 



(BS)i = minmax > P i Au,v){g i j + SA. (39) 

U V * 4 "" " " 

3=1 

As an example of a nonlinear result we shall get here an analog of the result on the 
existence of the average income for long lasting games (see [H] for a proof). 

Proposition 6.1. If the mapping v is a contraction uniformly in u,v, i.e. if 

\\v(u,v,fi l ) - v{u,v^ 2 )\\ < 5H/1 1 - /i 2 || (40) 

with a 5 G (0, 1), where \\v\\ = ^27=1 \ v i\> an< ^ if 9 ^ s Lipschitz continuous, i.e. 

\\g(u,v,n l ) -g(u,v^ 2 )\\ < C\\/j} - fi 2 \\ (41) 

with a constant C > 0, then there exists a unique A £ R and a Lipschitz continuous 
function S on S n such that 

B(S) = X + S, (42) 

and for all g £ C(E n ) we have 

\\B m g-m\\\ < \\S\\ + \\S-g\\, (43) 
B m a 

lim = A. (44) 

m— ¥oc m 

One can extend the other results for stochastic multi-step games to this nonlinear 
setting, say, the turnpike theorems from [10J, and then go on studying the nonlinear 
Markov analogs of differential games. 
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7 Colored options as a game against Nature 



Here we introduce a game-theoretic analysis of rainbow options in incomplete markets. 
Further developments can be found in [12], [3] and [9]. 

Recall that a European option is a contract between two parties where one party 
has right to complete a transaction in the future (with previously agreed amount, date 
and price) if he/ she chooses, but is not obliged to do so. More precisely, consider a 
financial market dealing with several securities: the risk-free bonds (or bank account) and 
J common stocks, J = 1, 2.... In case J > 1, the corresponding options are called colored 
or rainbow options (J-colors option for a given J). Suppose the prices of the units of these 
securities, B m and S l m , i G {1, 2, J}, change in discrete moments of time m — 1,2, ... 
according to the recurrent equations B m+ i = pB m , where the p > 1 is an interest rate 
which remains unchanged over time, and S l m+1 = C m +iSm, where i G {1,2,..., J}, are 
unknown sequences taking values in some fixed intervals Mj = C R. This model 

generalizes the colored version of the classical CRR model in a natural way. In the latter 
a sequence is confined to take values only among two boundary points di,Ui, and it 
is supposed to be random with some given distribution. In our model any value in the 
interval [c^-Uj] is allowed and no probabilistic assumptions are made. 

The type of an option is specified by a given premium function / of J variables. The 
following are the standard examples: 

option delivering the best of J risky assets and cash 

f(S\ S 2 , S J ) = max(S\ S 2 , 5 J , K), (45) 
calls on the maximum of J risky assets 

f{S\S 2 ,...,S J ) = max(0, max(S' 1 , S 2 ,..., S J ) — K), (46) 
multiple-strike options 

f(S\ S 2 , S J ) = max(0, S 1 - K u S 2 - K 2 , S J - Kj), (47) 
portfolio options 

f{S\ S 2 ,..., S J ) = max(0, mS 1 + n 2 S 2 + ... + njS J - K), (48) 
and spread options 

f{S\S 2 ) = max(0,(5 2 - S 1 ) - K). (49) 

Here, the S 1 , S 2 , S J represent the (in principle unknown at the start) expiration 
date values of the underlying assets, and K, K\, Kj represent the (agreed from the 
beginning) strike prices. The presence of max in all these formulae reflects the basic 
assumption that the buyer is not obliged to exercise his/her right and would do it only in 
case of a positive gain. 

The investor is supposed to control the growth of his/her capital in the following way. 
Let X m denote the capital of the investor at the time m = 1, 2, .... At each time m— 1 the 
investor determines his portfolio by choosing the numbers 7^ of common stocks of each 
kind to be held so that the structure of the capital is represented by the formula 
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J J 

X m -1 = ^2lLSL-l + ( X m-1 -^2lLSL-l), 
3=1 3=1 

where the expression in brackets corresponds to the part of his capital held in the bank 
account. The control parameters 7^ can take all real values, i.e. short selling and bor- 
rowing are allowed. The value £ m becomes known at the moment m and thus the capital 
at the moment m becomes 

j j 

Xm = £t£&^1 +A>(*m-1 ~ ^T^-l), (50) 
3=1 3=1 

if transaction costs are not taken into account. 

If n is the prescribed maturity date, then this procedures repeats n times starting from 
some initial capital X = X (selling price of an option) and at the end the investor is 
obliged to pay the premium / to the buyer. Thus the (final) income of the investor equals 

G{X n , S*, S"^, ■ S^) = X n — f(S^, S^, ■ S^). (51) 

The evolution of the capital can thus be described by the n-step game of the investor 
with Nature, the behavior of the latter being characterized by unknown parameters ^ 3 m . 
The strategy of the investor is by definition any sequences of vectors (71, • • • , 7«) such that 
each 7 m could be chosen using the whole previous information: the sequences X , ...,X m _i 
and Sq, S ] m _ 1 (for every stock j = 1, 2, J). The control parameters 7^ can take all 
real values, i.e. short selling and borrowing are allowed. A position of the game at any 
time m is characterized by J + 1 non- negative numbers X m , S^, • • • , with the final 
income specified by the function 

G(X,S\...,S J ) = X-f(S\...,S J ). (52) 

The main definition of the theory is as follows. A strategy 71, • • • , 7„, of the investor 
is called a hedge, if for any sequence (£1, • • • ,£ n ) the investor is able to meet his/her 
obligations, i.e. 

G(X n , S^, S£) > 0. 

The minimal value of the capital X for which the hedge exists is called the hedging price 
H of an option. 

Looking for the guaranteed payoffs means looking for the worst-case scenario (so called 
robust- control approach), i.e. for the minimax strategies. Thus if the final income is 
specified by a function G, the guaranteed income of the investor in a one-step game with 
the initial conditions X, S 1 , S J is given by the Bellman operator 

1 j j 

BG(X, S 1 , ■ ■ ■ , S J ) = - max . min G(pX + V 7*^ -pV 7^, ^S 1 , • • • , £ J S J ), 

(53) 

and (as it follows from the standard backward induction argument) the guaranteed income 
of the investor in the n-step game with the initial conditions X , Sq, Sq is given by the 
formula 



18 



B n G(X , Sq, 5"o ). 
In our model G is given by ( J52l) . Clearly for G of the form 

G(X,S\--- ,S J ) = X-f(S\--- ,S J ), 

1 J 
BG(X, S\ S J ) = X - -minmaxt/^S 1 ,^ 2 , . . . - - p)], 

p 7 * ^ 

and hence 

B»G(X, 5 1 , ■ ■ • , S J ) = X - \(B n f)(S\ ■■■,S J ), 
where the reduced Bellman operator is defined as: 



(Bf)(z\...,z J ) = mm max [f(£V,£V, • ■ ■ ,£V) - - p)], (54) 

7 {fefeiy]} ^ 



i=i 



or, more concisely, 



= min r ^ax J/((oz) - (7,^0 (55) 

7 

This leads to the following result from [7]. 

Theorem 7.1. TTie minimal value of X for which the income of the investor is non- 
negative (and which by definition is the hedge price H n in the n-step game) is given by 

H n = ^{B n f){Sl--- 1 S J ). (56) 

We shall now develop a method for evaluating the operator f|54l showing in particular 
how naturally the risk- neutral probability laws appear, as if by miracle, in this evaluation. 
The proof of all results below (and its various extensions) can be found in [9]. 

Let us say that a probability law \i G V[E) on a compact set E C R d is risk-neutral 
with respect to the origin, or shortly, risk-neutral if the origin is its barycenter, that is 
f E £[i(d£) = 0. The set of all risk-neutral laws on E will be denoted by V rn (E). 

Let us say that a finite family of vectors E = {£1, • • • , in R d is in general position if 
the vectors of any subset of {£1, • • • of size d are linearly independent (in particular, 
all vectors in E are non- vanishing) . A subset E C H d is called (strongly) positively 
complete, if there exists no w € H d such that (u>, £) > for all £ e i£. 

Proposition 7.1. Lei a finite set E = {£1, ■ • • , fre strongly positively complete 
in H, d . Then the family E is in general position and there exists a unique risk-neutral 



probability law {pi, 


■■■ , P d^ 


-1} on 


• • • > £d 


4_i }, with 
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,d, (57) 
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where 

/l 1 ••• 1 \ 

C = det ^ & "' . (58) 

For arbitrary k we have the following. 

Proposition 7.2. Lei a family E = ■ • • 6e strongly positively complete and in 
general position. Then the extreme points of the convex set of risk-neutral probabilities 
on • • ■ are risk-neutral probabilities with supports on strongly positively complete 
subsets of E of size precisely d+ 1. 

Now we are going to evaluate the minimax expression 

• ■ ■ ,&](/) = inf max[/(&) - (&, 7 )]. (59) 

Theorem 7.2. Lei a family of vectors £i, • • • , m R d fre strongly positively complete 
and in general position. Then 

n[6,-- - ,&](/) = max E„/(£), (60) 

where max zs taken over all extreme points p of risk-neutral laws on • ■ ■ <?2fen 
fry Proposition 7.2, and inf m ( 159"]) zs attained on a certain finite 7. 

Changing variables £ = . . . , £ J ) to r\ = £ o z yields 

1 J 
(Bf)(z\...,z J ) = -nun max [/fa) - - p^)], (61) 



7 {^e^di^^j]} 

1=1 



or, by shifting, 



0B/)(z\...,z J ) = -min max [f^ + pz)-^^)]. (62) 

p 7 {»ye^ l (*-p),2 l K-p)]} 

Assuming / is convex (which is often the case for option payoffs), we can apply Theorem 
17.21 where max is taken over the set of vectors 

rjj = fa o z - pz, 
being the vertices of the rectangular parallelepiped 

n,, p = x f =1 [z* (di -p),z { { Ui -p)], 

where 

£r = {di\iei, Ujlj^}, 

are the vertices of 

IL=xj =1 [d i ,u i ], (63) 

parametrized by all subsets (including the empty one) Id {1, . . . , J}. 

Since the origin is an internal point of II (because di < p < ui), the family {77/} is 
strongly positively complete. The condition of general position is rough in the sense that 
it is fulfilled for an open dense subset of pairs (di,ui). Applying Theorem 17.21 to ( 1621) and 
returning back to £ yields the following. 
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Theorem 7.3. If the vertices £/ of the parallelepiped II are in general position in the 
sense that for any J subsets Ii, • • • , Ij, the vectors {£4 — pl} J k=1 are independent in R J , 
then 

(S/)(z) = -maxE n /(£oz), z = (z\ ■ ■ ■ , z J ), (64) 

p m 

where {Q} is the collection of all subsets Q — £4, • • ■ , £/ J+1 0/ £/ie set 0/ vertices of II, 0/ 
sue J + 1, sitc/j t/iat their convex hull contains pi as an interior point (1 is the vector 
with all coordinates 1), and where denotes the expectation with respect to the unique 
probability law {pi}, £7- G fi, on the set of vertices of II, which is supported on Q and is 
risk-neutral with respect to pi, that is 

M/ = pl- (65) 

/C{1,...,J} 

Risk-neutrality now corresponds to its usual meaning in finance, i.e. (16"5|) means that 
all discounted stock prices are martingales. 

Notice that the max in fl64l) is over a finite number of explicit expressions, which 
is of course a great achievement as compared with initial minmax over an infinite set. 
In particular, it reduces the calculation of the iterations B n f to the calculation on a 
controlled Markov chain. Let us also stress that the number of eligible Q in (|64|) is the 
number of different pyramids (convex polyhedrons with J+l vertices) with vertices taken 
from the vertices of II and containing pi as an interior point. Hence this number can be 
effectively calculated. 

Let us point our some properties of the operator B given by that are obvious, but 
important for practical calculations: pB is non-expansive: 

\\B{h)-B{f 2 )\\<-\\h-hl 
p 

and homogeneous (both with respect to addition and multiplication): 

P B(X + f) = X + pB(f), B(Xf) = XB(f) 

for any function / and A G R (resp. A > 0) for the first (resp second) equation. 
Next, if f p is a power function, that is 

/ P (s) = CO* 1 •■■ 

then / p (f oz) = f P (0f P ( z ), implying 

(B n f p )(z) = ((Bf p )(l)Tf p (z). (66) 

Therefore, power functions are invariant under B (up to a multiplication by a constant). 
Consequently, if for a payoff / one can find a reasonable approximation by a power 
function, that is there exists a power function f p such that ||/ — f p \\ < e, then 

\\B n f-X n f P \\<^\\f-f P \\<^ X = (Bf p )(l), (67) 
p n p n 

so that an approximate calculation of B n f is reduced to the calculation of one number A. 
This implies the following scheme for an approximate evaluation of B: first find the best 
fit to / in terms of functions a + f p (where f p is a power function and a a constant), and 
then use ( |67j) . 
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